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" The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )K Responsive to connmunication(s) filed on 29 March 2001 . 
2a)n This action is FINAL. 2b)^ This action is non-final. 

3) n Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) ^ Claim(s) 1-23 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 

6) [E1 Claim(s) 1-3,6-10 and 13-23 is/are rejected. 

7) 13 Claim(s) 4,5,11 and 12 is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) IEI The specification is objected to by the Examiner. 

10)^ The drawing(s) filed on 29 March 2001 is/are: 8)0 accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
1 1 )Q The proposed drawing correction filed on is: a)^ approved b)^ disapproved by the Examiner. 

If approved, corrected drawings are required In reply to this Office action. 

12) 0 The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) 13 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)[ElAII b)n Some*c)n None of: 

1 Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (POT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) n Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 11 9(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
Attachment(s) 

1 ) ^ Notice of References Cited (PTO-892) 4) □ Interview Summary {PTO-41 3) Paper No(s). . 

2) n Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) [D Notice of Informal Patent Application (PTO-152) 

3) S Information Disclosure Statement(s) (PTO-1449) Paper No(s) 4J_- 6) □ Other: 
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DETAILED ACTION 



1 . Because the language of certain claims is written in the manner prescribed to 
determine the equivalents of the element, as required by 35 U.S.C. 112, 6*^ paragraph, 
but the specification does not include listing the where indicated, the Examiner is 
proceeding with the understanding that such claims are not intended to be examined as 
"plus function" claims and that the term "means" indicates "process". See MPEP 
§2111.01 [R-1] &§2181-§2186 that reads, in part: "... the specification must be 
consulted to determine the structure, material, or acts corresponding to the function 
recited in the claim. " 

(Note that this application has been included in Art Unit 2654, and that this AU 
number should be used in all future correspondence.) 



2. The information disclosure statements filed 15 June, 09 May and 29 March, 2001 
fail to comply with 37 CFR 1 .98(a)(1 ) and (b), which require a list of all patents, 
publications, or other information submitted for consideration by the Office. 

- Applications 09/037,998, 09/263,262, 09/264,866. 09/262,852 are appended to the 
form PTO-892 accompanying this action and have been considered. 

- Application 09/301 ,669 was not received and is not included in the 892 as the 
information referred to therein has not been considered. See MPEP § 609 A(1). 



Information Disclosure Statement 
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Drawings 



3. The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(5) 
because they include the following reference sign(s) not mentioned in the description: 

- In figure 7, "S602" is not specified. 

- A proposed drawing correction, corrected drawings, or amendment to the 
specification to add the reference sign(s) in the description, are required in reply to 
the Office action to avoid abandonment of the application. The objection to the 
drawings will not be held in abeyance. 



4. The specification is objected to as failing to provide proper antecedent basis for 
the claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01 (o). Correction 
of the following is required: 

- The term ''upper liklihoods" found in claims 6 and 13 (lines 23 & 1, respectively) is 
not explained in the specification. 

- The Examiner is proceeding with the understanding that the term is not of 
significance being employed in conjunction with another limit on the same feature 
that was defined. 



Specification 
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5. The title of the invention is not descriptive. A new title is required that is clearly 
indicative of the invention to which the claims are directed. 
The following title is suggested: 

"Speech signal segment recognition dictionary construction and maintenance". 

Claim Objections 



6. Claim 1 7 is objected to because of the following informalities: 
- "Learning" is misspelled (2"^ line claim 17). 

Appropriate correction is required. 

Claim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which that subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

8. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 1 03(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
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under 37 CFR 1 .56 to point out the inventor and invention dates of each clainn that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 

Maddox & Komori et al'^^^^'^^^ 

9. Claims 1.6.8, 13, 15, 17, 18,20,21 and 23 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Maddox (U.S. Patent 6,000,024 A) in view of Komori et af^^^ 
(U.S. Patent 5.812.975) and further in view of Komori a/^^^ (U.S. Patent 5,787,396). 



10. Regarding claims 1 and 8, the invention for a parallel computing system by 
Maddox reads on the feature of the claim for a speech signal processing apparatus as 
follows: 

- Maddox (column 6 lines 44-46) reads on the feature of a speech segment search 
means for searching a speech database for speech segments that satisfy a phonetic 
environment; 

' Maddox does not mention HMM. The state transition model design and voice 
recognition of Komori et al^^^ (column 3 lines 32-38) reads on the feature of HMM 
learning means for computing HMMs of phonemes on the basis of a search result of 
that speech segment search means and (column 6 lines 52-53) reads on the feature 
of segment recognition means for performing segment recognition of the speech 
segments on the basis of the HMMs of the phonemes; 
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- Maddox does not mention registering segments. The speech recognition method of 
Komori et al'^^^ (column 4 lines 24-30) reads on the feature of registration segment 
determination means for determining a speecii segment to be registered in a 
segment dictionary in accordance witti a segment recognition result of that segment 
recognition means. 

- It would have been obvious to a person of ordinary skill in the art of speech signal 
processing at the time of the invention to apply the method/teachings of Komori et al 
to the device/method of Maddox so that differences in phone environments - such 
as preceding and succeeding phones - can be classified finely. 

11. Regarding claims 6 and 13 as understood by the Examiner; the claims are set 
forth with the same limits as claims 1 and 8, respectively. Maddox does not mention 
registering segments. Komori et ai^^^ (2032 in figure 8) reads on the feature of speecti 
segments liaving iketilioods not less than a predetermined value registered in the 
segment dictionary which would have made it obvious to a person of ordinary skill in the 
art of speech signal processing at the time of the invention to apply the method and/or 
teachings of Komori et al"^^^ to the device/method of Maddox so as to have recognition 
results reflect an acceptable comparison. 

1 2, Regarding claim 1 5, the claim is set forth with the same limits as claim 8. 
Maddox (claim 7 line 32) reads on the feature of a computer readable storage medium 
storing a program for implementing a method cited. 
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1 3. Regarding claims 1 7 and 20, the features of the claims are the same as those 
found in claims 1 and 8 and the claims are rejected for the same reasons. 

14. Regarding claims 1 8 and 21 ; the claims are set forth with the same limits as 
claims 17 and 20, respectively. Maddox is silent as to use of HMM. Komori et al"^^^ 
(column 4 lines 15-31) reads on the feature that obtains a maximum likelihood HMM 
which has a maximum likelihood with one of the plurality of speech segments from the 
HMMs corresponding to the phonemes, checks if the one speech segment is a speech 
segment used in learning of the maximum likelihood HMM, and selects the one speech 
segment when the one speech segment is a speech segment used in learning of the 
maximum likelihood HMM. It would have been obvious to a person of ordinary skill in 
the art of speech signal processing at the time of the invention to apply the method 
and/or teachings of Komori al to the device/method of Maddox so as to process faster 
without reducing recognition accuracy. 

1 5. Regarding claim 23, the claim is set forth with the same limits as claim 20. 
Maddox (claim 7 line 32) reads on the feature of a computer readable storage medium 
storing a program for implementing a method cited. 
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Maddox. Komori et al'^^^^'^^\ & Rosenberg 

1 6. Claims 2 and 9 are rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Maddox in view of Komori et af^^^ and further in view of Komori et ar^^^ and further in 
view of Rosenberg ("Connected Sentence Recognition Using Diphone-Like Templates" 
International Conference on Acoustics, Speech, and Signal Processing, April 1988). 

17. Regarding claims 2 and 9; the claims are set forth with the same limits as claims 
1 and 8, respectively. Maddox is silent on the feature of diphones or biphones. 
Rosenberg (lines 7-9 left column page 473) reads on the feature that segment 
recognition means adopts diphones as units of the phonemes, categorizes speech 
segments into four categories CC, CV, VC, and VV (C: a consonant, V: a vowel), and 
performs segment recognition in each category (in the end of that paragraph). It would 
have been obvious to a person of ordinary skill in the art of speech signal processing at 
the time of the invention to apply the method/teachings of Rosenberg to the device 
and/or method of Maddox because defining units larger than phones will be easier to 
work with, segment and label by virtue of containing greater variations and context 
effects. 

Maddox, Komori ef a/^^^^ ^^^^ & Tominaga 

18. Claims 3 and 10 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Maddox in view of Komori et af^^^ and further in view of Komori et al'^^^ and further in 
view of Tominaga (U.S. Patent 5,31 1 ,429 A). 



Application/Control Number: 09/819,613 
Art Unit: 2654 



Page 9 



1 9. Regarding claims 3 and 1 0; the claims are set forth with the same limits as 
claims 1 and 8, respectively. Maddox does not mention registering segments. 

The speech recognition method of Komori et a/^^^ (column 4 lines 24-30) reads 
on the feature of determining a speech segment to be registered in a segment 
dictionary in accordance with a segment recognition result of that segment recognition. 

The maintenance support method and apparatus for natural language processing 
system of Tominaga (column 1 1 lines 19-36) reads on the feature that checks if a 
speech segment pattern which matches a speech segment that is not successfully 
recognized by that segment recognition means, and registers that speech segment in 
the segment dictionary if the corresponding speech segment pattern is found. 

It would have been obvious to a person of ordinary skill in the art of speech 
signal processing at the time of the invention to apply the method/teachings of Komori 
ef a/ & Tominaga to the device/method of Maddox so as to avoid duplicate entries. 

Maddox, Komori et al^^^^'^^\ & Richardson etal 

20. Claims 7 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Maddox in view of Komori et af^^^ and further in view of Komori et a/^^^ and further in 
view of Richardson ef al (U.S. Patent 5,926,784 A). 

21 . Regarding claims 7 and 14; the claims are set forth with the same limits as 
claims 6 and 13, respectively. Maddox does not mention registering segments. 
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In natural language parsing using Podding, Richardson et al (column 1 lines 21- 
25) reads on the feature that registers, in the segment dictionary, speech segments 
having upper values (claims 39 and 41 ) obtained by normalizing the likelihoods by 
durations of the speech segments or likelihoods having the values not less than a 
predetermined value. 

It would have been obvious to a person of ordinary skill in the art of speech 
signal processing at the time of the invention to apply the method/teachings of 
Richardson et al to the device/method of Maddox so as to further improve the efficiency 
with probabilities assigned to the syntax rules. 

Maddox, Komori et al^^^^'^^^, Tominaga, Fukada etaL Huang etal^^^ 

22. Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over Maddox 
in view of Komori et al^^^ and further in view of Komori al"^^^ and further in view of 
Tominaqa and further in view of Fukada et al (U.S. Patent 5,845,047 A) and further in 
view of Huang et a/^^^ (U.S. Patent 5,913,193 A) 

23. Regarding claim 16, the claim is set forth with the same limits as claim 8. The 
feature of a segment dictionary in which speech segments are registered is cited in 
claim 8 and the rejections are applied to this claim for the same reasons. 

Where Maddox is silent on the subject of language analysis, Tominaga (column 3 
lines 46-56) reads on the feature for performing language analysis of input text data. 
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Where Maddox is silent on the subject of prosody, Fukada et al (column 6 line 
59) reads on the feature of generating prosody on the basis of an analysis result of that 
language analysis means and the runtime acoustic unit selection for speech synthesis 
of Huang et af^^^ (column 3 lines 1 -4) reads on the feature that search that segment 
dictionary on the basis of the prosody generated fey that prosody generation means to 
select corresponding speech segments and, with (column 1 line 66) reads on modifying 
and concatenating the speech segments selected by that speech segment selection 
means and (132 in figure 5) for reproducing speech on the basis of the result modified 
by that speech segment modification/concatenation means. 

It would have been obvious to a person of ordinary skill in the art of speech 
signal processing at the time of the invention to apply the method/teachings of 
Tominaga, Fukada et al & Huang et aP^^ to the device/method of Maddox so as to 
improve the quality of synthetic speech by considering additional attributes of language. 

Maddox, Komori etal^^^^'^^\ & Huang etal'^^^ 

24. Claims 19 and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Maddox in view of Komori et aP^^ and further in view of Komori et al'^^^ and further 
in view of Huang etaP^U U.S. Patent 5,913,193 A). 

25. Regarding claims 19 and 22; the claims are set forth with the same limits as 
claims 17 and 20, respectively. Maddox is silent as to speech synthesis, Huang et 
aP^^ (36 in figure 1 ) reads on the feature of producing synthetic speech using the 
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segment dictionary (22^->36 in figure 1 ) ... which would have made it obvious to a 
person of ordinary skill in the art of speech signal processing at the time of the invention 
to apply the method/teachings of Huang al to the device/method of Maddox so as to 
generate natural-sounding speech. 



26. Claims 4-5 and 11-12 are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

27. The following is a statement of reasons for the indication of allowable subject 



- The present invention is directed to building and maintaining speech segment 
dictionaries. 

- Claims 4 and 1 1 identify the uniquely distinct feature that "registers a speech 
segment in tiie segment dictionary when the number of speech segments 
recognized is not less than a predetermined value , " 

The closest prior art, Yokota et al , discloses registering recognized speech 
segments but fails to anticipate or render the above underlined limitations obvious. 

- Claims 5 and 12 depend on claims that were found to be allowable and so would 
they be allowed as a consequence. 



Allowable Subject Matter 



matter: 
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28. As allowable subject matter has been indicated, applicant's reply must either 
comply with all formal requirements or specifically traverse each requirement not 
complied with. See 37 CFR 1 . 1 1 1 (b) and MPEP § 707.07(a). 

Conclusion 

The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

- Nakagawa et al ("A Method For Continuous Speech Segmentation Using HMM", 9' 
International Conference on Pattern Recognition, November 1988) statistical 
segmentation using a hidden Markov model (HMM) of vowels & consonants. 

- Yokota et al (U.S. Patent 6,385,339 B1 ) collaborative learning system and pattern 
recognition method. 

- Tominaga (U.S. Patent 5,31 1 ,429 A) maintenance support method and apparatus 
for natural language processing system. 

- Huang et al'^^^ (U.S. Patent 5,905,972 A) prosodic databases holding fundamental 
frequency templates for use in speech synthesis. 

29. Any inquiry concerning this communication or earlier communications from the 
Examiner should be directed to Daniel A. Nolan at telephone (703) 305-1368 whose 
normal business hours are Mon, Tue, Thu & Fri, from 7 AM to 5 PM. 
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If attempts to contact the examiner by telephone are unsuccessful, supervisor 
Richemond Dorvil can be reached at (703)305-9645. 

The fax phone number for Technology Center 2600 is (703)872-9314. Label 
informal and draft communications as "DRAFT" or "PROPOSED", & designate formal 
communications as "EXPEDITED PROCEDURE". Formal response to this action may 
be faxed according to the above instructions, 

or mailed to: Commissioner of Patents and Trademarks 



Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to Technology Center 2600 Customer Service Office at 
telephone number (703) 306-0377. 



Washington, D.C. 20231 



or hand-delivered to: 



Crystal Park 2, 

2121 Crystal Drive, Arlington, VA, 
Sixth Floor (Receptionist). 



Daniel A. Nolan 
Examiner 
Art Unit 2654 



DAN/d 

October 6, 2003 




